Multimodal machine learning for intelligent mobility

Scientific problems are solved by finding the optimal solution for a specific task. Some problems can be solved analytically while other problems are solved using data driven methods. The use of digital technologies to improve the transportation of people and goods, which is referred to as intelligent mobility, is one of the principal beneficiaries of data driven solutions. Autonomous vehicles are at the heart of the developments that propel Intelligent Mobility. Due to the high dimensionality and complexities involved in real-world environments, it needs to become commonplace for intelligent mobility to use data-driven solutions. As it is near impossible to program decision making logic for every eventuality manually. While recent developments of data-driven solutions such as deep learning facilitate machines to learn effectively from large datasets, the application of techniques within safety-critical systems such as driverless cars remain scarce.Autonomous vehicles need to be able to make context-driven decisions autonomously in different environments in which they operate. The recent literature on driverless vehicle research is heavily focused only on road or highway environments but have discounted pedestrianized areas and indoor environments. These unstructured environments tend to have more clutter and change rapidly over time. Therefore, for intelligent mobility to make a significant impact on human life, it is vital to extend the application beyond the structured environments. To further advance intelligent mobility, researchers need to take cues from multiple sensor streams, and multiple machine learning algorithms so that decisions can be robust and reliable. Only then will machines indeed be able to operate in unstructured and dynamic environments safely. Towards addressing these limitations, this thesis investigates data driven solutions towards crucial building blocks in intelligent mobility. Specifically, the thesis investigates multimodal sensor data fusion, machine learning, multimodal deep representation learning and its application of intelligent mobility. This work demonstrates that mobile robots can use multimodal machine learning to derive driver policy and therefore make autonomous decisions.To facilitate autonomous decisions necessary to derive safe driving algorithms, we present an algorithm for free space detection and human activity recognition. Driving these decision-making algorithms are specific datasets collected throughout this study. They include the Loughborough London Autonomous Vehicle dataset, and the Loughborough London Human Activity Recognition dataset. The datasets were collected using an autonomous platform design and developed in house as part of this research activity. The proposed framework for Free-Space Detection is based on an active learning paradigm that leverages the relative uncertainty of multimodal sensor data streams (ultrasound and camera). It utilizes an online learning methodology to continuously update the learnt model whenever the vehicle experiences new environments. The proposed Free Space Detection algorithm enables an autonomous vehicle to self-learn, evolve and adapt to new environments never encountered before. The results illustrate that online learning mechanism is superior to one-off training of deep neural networks that require large datasets to generalize to unfamiliar surroundings. The thesis takes the view that human should be at the centre of any technological development related to artificial intelligence. It is imperative within the spectrum of intelligent mobility where an autonomous vehicle should be aware of what humans are doing in its vicinity. Towards improving the robustness of human activity recognition, this thesis proposes a novel algorithm that classifies point-cloud data originated from Light Detection and Ranging sensors. The proposed algorithm leverages multimodality by using the camera data to identify humans and segment the region of interest in point cloud data. The corresponding 3-dimensional data was converted to a Fisher Vector Representation before being classified by a deep Convolutional Neural Network. The proposed algorithm classifies the indoor activities performed by a human subject with an average precision of 90.3%. When compared to an alternative point cloud classifier, PointNet[1], [2], the proposed framework out preformed on all classes. The developed autonomous testbed for data collection and algorithm validation, as well as the multimodal data-driven solutions for driverless cars, is the major contributions of this thesis. It is anticipated that these results and the testbed will have significant implications on the future of intelligent mobility by amplifying the developments of intelligent driverless vehicles.</div

Robust Fusion of LiDAR and Wide-Angle Camera Data for Autonomous Mobile Robots

Autonomous robots that assist humans in day to day living tasks are becoming increasingly popular. Autonomous mobile robots operate by sensing and perceiving their surrounding environment to make accurate driving decisions. A combination of several different sensors such as LiDAR, radar, ultrasound sensors and cameras are utilized to sense the surrounding environment of autonomous vehicles. These heterogeneous sensors simultaneously capture various physical attributes of the environment. Such multimodality and redundancy of sensing need to be positively utilized for reliable and consistent perception of the environment through sensor data fusion. However, these multimodal sensor data streams are different from each other in many ways, such as temporal and spatial resolution, data format, and geometric alignment. For the subsequent perception algorithms to utilize the diversity offered by multimodal sensing, the data streams need to be spatially, geometrically and temporally aligned with each other. In this paper, we address the problem of fusing the outputs of a Light Detection and Ranging (LiDAR) scanner and a wide-angle monocular image sensor for free space detection. The outputs of LiDAR scanner and the image sensor are of different spatial resolutions and need to be aligned with each other. A geometrical model is used to spatially align the two sensor outputs, followed by a Gaussian Process (GP) regression-based resolution matching algorithm to interpolate the missing data with quantifiable uncertainty. The results indicate that the proposed sensor data fusion framework significantly aids the subsequent perception steps, as illustrated by the performance improvement of a uncertainty aware free space detection algorith

IoT driven ambient intelligence architecture for indoor intelligent mobility

Personal robots are set to assist humans in their daily tasks. Assisted living is one of the major applications of personal assistive robots, where the robots will support health and wellbeing of the humans in need, especially elderly and disabled. Indoor environments are extremely challenging from a robot perception and navigation point of view, because of the ever-changing decorations, internal organizations and clutter. Furthermore, human-robot-interaction in personal assistive robots demands intuitive and human-like intelligence and interactions. Above challenges are aggravated by stringent and often tacit requirements surrounding personal privacy that may be invaded by continuous monitoring through sensors. Towards addressing the above problems, in this paper we present an architecture for "Ambient Intelligence" for indoor intelligent mobility by leveraging IoTs within a framework of Scalable Multi-layered Context Mapping Framework. Our objective is to utilize sensors in home settings in the least invasive manner for the robot to learn about its dynamic surroundings and interact in a human-like manner. The paper takes a semi-survey approach to presenting and illustrating preliminary results from our in-house built fully autonomous electric quadbike

The Function Acquisition Speed Test (FAST) as a Measure of Verbal Stimulus Relations in the Context of Condom Use

The Function Acquisition Speed Test (FAST) has shown recent evidence as an effective tool for the quantification of stimulus relatedness. The current study assessed the potential of the FAST in measuring the effects of the presentation of positively or negatively valenced messages on relatedness between stimulus relations with regard to safe-sex behavior, namely condom use. Fifty-one participants were assigned to one of three conditions comprised of valenced message interventions regarding the impact of condom use on the enjoyment of sexual behavior (each condition n = 17): a positive-message condition, a negative-message condition, or a no-message control condition. A significant Strength of Relation (SoR) score was found across positive and negative FAST test trials in the positive-message condition only, with no significant differences in SoR scores observed for either the Negative-message or Control conditions. These data suggest that the FAST may have utility as a sensitive behavioral tool for measuring changes in stimulus relations concerning safe-sex behavior on the basis of brief message interventions

Genetic stock identification of Atlantic salmon (Salmo salar) populations in the southern part of the European range

notes: PMCID: PMC2882343© 2010 Griffiths et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Anadromous migratory fish species such as Atlantic salmon (Salmo salar) have significant economic, cultural and ecological importance, but present a complex case for management and conservation due to the range of their migration. Atlantic salmon exist in rivers across the North Atlantic, returning to their river of birth with a high degree of accuracy; however, despite continuing efforts and improvements in in-river conservation, they are in steep decline across their range. Salmon from rivers across Europe migrate along similar routes, where they have, historically, been subject to commercial netting. This mixed stock exploitation has the potential to devastate weak and declining populations where they are exploited indiscriminately. Despite various tagging and marking studies, the effect of marine exploitation and the marine element of the salmon lifecycle in general, remain the "black-box" of salmon management. In a number of Pacific salmonid species and in several regions within the range of the Atlantic salmon, genetic stock identification and mixed stock analysis have been used successfully to quantify exploitation rates and identify the natal origins of fish outside their home waters - to date this has not been attempted for Atlantic salmon in the south of their European range.European Union INTERREG IIIB programme (Atlantic Salmon Arc Project [ASAP], Project No. 040)

Graphene-porphyrin single-molecule transistors

We demonstrate a robust graphene-molecule-graphene transistor architecture. We observe remarkably reproducible single electron charging, which we attribute to insensitivity of the molecular junction to the atomic configuration of the graphene electrodes. The stability of the graphene electrodes allow for high-bias transport spectroscopy and the observation of multiple redox states at room-temperature

Can grapheme-color synesthesia be induced by hypnosis?

Grapheme-color synesthesia is a perceptual experience where graphemes, letters or words evoke a specific color, which are experienced either as spatially coincident with the grapheme inducer (projector sub-type) or elsewhere, perhaps without a definite spatial location (associator sub-type). Here, we address the question of whether synesthesia can be rapidly produced using a hypnotic color suggestion to examine the possibility of “hypnotic synesthesia”, i.e., subjectively experienced color hallucinations similar to those experienced by projector synesthetes. We assess the efficacy of this intervention using an “embedded figures” test, in which participants are required to detect a shape (e.g., a square) composed of local graphemic elements. For grapheme-color synesthetes, better performance on the task has been linked to a higher proportion of graphemes perceived as colored.We found no performance benefits on this test when using a hypnotic suggestion, as compared to a no-suggestion control condition. The same result was found when participants were separated according to the degree to which they were susceptible to the suggestion (number of colored trials perceived). However, we found a relationship between accuracy and subjective reports of color in those participants who reported a large proportion of colored trials: trials in which the embedded figure was accurately recognized (relative to trials in which it was not) were associated with reports of more intense colors occupying a greater spatial extent. Collectively, this implies that hypnotic color was only perceived after shape detection rather than aiding in shape detection via color-based perceptual grouping. The results suggest that hypnotically induced colors are not directly comparable to synesthetic ones

Implementation Science Workshop: Primary Care-Based Multidisciplinary Readmission Prevention Program

Editor's note: In this installment of Implementation Science Workshop, Dr. Cavanaugh and colleagues describe implementation and evaluation of a readmission prevention program at the University of North Carolina. In an accompanying commentary, Elizabeth Yano of the West Los Angeles VA highlights strengths and weaknesses of their approach, emphasizing generalizable lessons for learning health care organizations. -- Lindsay Jubelt, MD, MS, Contributing Edito